Color correction of uncalibrated images for the classification of human skin color

Images of a scene captured with multiple cameras have different color values due to variations in capture and color rendering across devices. We present a method to accurately retrieve color information from uncalibrated images taken under uncontrolled lighting conditions with an unknown device and no access to raw data, but with a limited number of reference colors in the scene. The method is used to assess skin tones for cosmetics recommendations. A subject is imaged with a calibration target in the scene. This target is extracted and its color values are used to compute a color correction transform that is applied to the entire image. We establish that the best mapping is done using a target consisting of skin colored patches representing a range of human skin colors. We show that color information extracted from images is well correlated with color data derived from spectral measurements of skin. Skin color can be consistently assessed across cameras with different color rendering and resolutions ranging from 0.1 Mpixels to 4.0 Mpixels. Introduction Our goal is to retrieve accurate color information from uncalibrated images taken with unknown cameras. Due to incomplete illuminant compensation and to the different characteristics of available cameras, consistent color rendering is not achieved and the same object captured with different cameras have different pixel values in the resulting images. Consequently, the retrieval of accurate color information requires either the pre-calibration of the imaging devices and the control of the illuminant, or additional scene information. We are interested in retrieving colorimetric as opposed to spectral information about the scene. Thus, to use any consumer camera as a colorimeter, we need known color information present in the scene in the form of a calibration target, whose pixel values are used as reference for the color correction of images. The method we present is developed for the color correction and classification of skin tones, but can be generalized to other colors. The appearance of skin has been studied mostly for rendering purposes in computer graphics, for face detection and tracking in computer vision, for diagnostic purposes in dermatology, and for makeup and skin care in cosmetics. However, there is little research on how to assess skin tones accurately from digital images. Perceived color is the most discriminative of skin attributes and depends on its pigmentation, blood microcirculation, roughness, sebum, and perspiration [2]. Its objective measurement has been made mostly by traditional reflectance spectrometry (for a review see [5]) and using narrow band spectrometers developed specifically for dermatology [8]. Spectrometry of skin has two Figure 1. uncorrected images (top row) and corrected images (bottom row) for cameras (from left to right) Canon S400, HP850, Nikon D1, Nokia 6820 main drawbacks: the area measured is about 0.05 cm2, but skin is not homogeneous [2]. Additionally, the pressure of the probe on the skin can be an important source of bias [7]. Still, traditional spectrometers are inexpensive and simple to use and thus widely employed. To overcome the problem of the probe pressure on skin, a proprietary device composed of an integrating sphere, a spectrometer, and a tri-CCD camera was developed [3], allowing non contact spectroscopy of different parts of the face and simultaneous imaging for estimation of the skin color inhomogeneity. Due to uneven tan, blemish, or shine, the color and appearance of skin are usually not uniform across a subject’s face. Moreover, because of its volume, there are also important shadows and specularities across the face, making the estimation of skin color from images more difficult. We present a simple and inexpensive method to assess skin color from digital images for applications such as online shopping, for which the use of calibrated devices is not feasible, or for automated suggestion of personal appearance products, such as makeup or clothing that complement skin tone. Uncalibrated images taken under unknown illuminants are color corrected by mapping selected pixel values onto reference values present in the scene in the form of a target. This target is extracted and a 3×4 linear color transform is computed by least mean square error between the extracted target color values and pre-computed color values. This color correction transform is then applied to the entire image and face pixel values extracted. Figure 1 shows the results of our method applied to images of the same subject captured with four different cameras. We show that our method allows color correcting skin tones with an accuracy in terms of CIELAB color difference of ∆E∗ ab < 1. Face color values extracted from color corrected images show